The XSEDE-funded Large Scale Video Analytics (LSVA) project addresses obstacles in image-retrieval and research using extreme-scale archives of video data. Deployed on the SDSC Gordon Compute Cluster, the LSVA includes the creation of visualization tools that enhance research in several ways: novel visualizations employ spatial and temporal simultaneity, revealing unique aspects of a single film sequence; comparative visualizations represent relationships among multiple films within an archive; and, finally, the integration of visualization imagery becomes an input tag and a front end process that feeds the Medici content management system and enhances word-based labels, helping to close the semantic gap that occurs when words are applied to images.
The technique developed to generate spatial representations of video is demonstrated in this animation. Custom software converts a digital video sequence into a three-dimensional dataset, termed a movie-cube, by extracting and ordering each frame of the sequence along the Z axis. Once in this form, a variety of techniques commonly used in scientific visualization can be used to show multiple perspectives, allowing analysis of image-based, time-based media that is unattainable in traditional methodologies. A custom visualization system is used to render and examine the dataset as shown in the animation by first animating a slice plane along the Z axis representing time within the video sequence. As expected, such movement within the movie-cube reveals the original movie sequence. Experimenting with different orientations of the slice plane, however, reveals unique and interesting patterns showing various aspects of the time-based data within a single spatial representation. Note, for example, how such visualizations provide a clear representation of cinematic elements such as camera shots, angles and movements.